A framework for relating natural movement to length and quality of life in human and non-human animals.

Natural movement is clearly related to health, however, it is also highly complex and difficult to measure. Most attempts to measure it focus on functional movements in humans, and while this a valid and popular approach, assays focussed on particular movements cannot capture the range of natural movement that occurs outside them. It is also difficult to use current techniques to compare movement across animal species. Interspecies comparison may be useful for identifying conserved biomechanical and/ or computational principles of movement that could inform human and veterinary medicine, plus several other fields of research. It is therefore important that research develops a system for quantifying movement in freely moving animals in natural environments and relating it to length and quality of life (LQOL). The present text proposes a novel theoretical framework for doing so, based on screening movement ability (MA). MA is calculated from three major variables - Movement Quality, Movement Complexity, and Movement Quantity. These may represent the most important components of movement as it relates to LQOL, and offer insight into how and why differences in the relationship between movement and LQOL occur. A constrained version of the framework is validated in Drosophila, which suggests that MA may indeed represent a useful new paradigm for understanding the relationship between movement and length and quality of life.

Behavioral Motor Performance.

The human sensorimotor control system has exceptional abilities to perform skillful actions. We easily switch between strenuous tasks that involve brute force, such as lifting a heavy sewing machine, and delicate movements such as threading a needle in the same machine. Using a structure with different control architectures, the motor system is capable of updating its ability to perform through our daily interaction with the fluctuating environment. However, there are issues that make this a difficult computational problem for the brain to solve. The brain needs to control a nonlinear, nonstationary neuromuscular system, with redundant and occasionally undesired degrees of freedom, in an uncertain environment using a body in which information transmission is subject to delays and noise. To gain insight into the mechanisms of motor control, here we survey movement laws and invariances that shape our everyday motion. We then examine the major solutions to each of these problems in the three parts of the sensorimotor control system, sensing, planning, and acting. We focus on how the sensory system, the control architectures, and the structure and operation of the muscles serve as complementary mechanisms to overcome deviations and disturbances to motor behavior and give rise to skillful motor performance. We conclude with possible future research directions based on suggested links between the operation of the sensorimotor system across the movement stages. © 2024 American Physiological Society. Compr Physiol 14:5179-5224, 2024.

Direction-Specific Effects of Artificial Skin-Stretch on Stiffness Perception and Grip Force Control.

When interacting with an object, we use kinesthetic and tactile information to create our perception of the object's properties and to prevent its slippage using grip force control. We previously showed that applying artificial skin-stretch together with, and in the same direction as, kinesthetic force increases the perceived stiffness. Here, we investigated the effect of the direction of the artificial stretch on stiffness perception and grip force control. We presented participants with kinesthetic force together with negative or positive artificial stretch, in the opposite or the same direction of the natural stretch due to the kinesthetic force, respectively. Our results showed that artificial skin-stretch in both directions augmented the perceived stiffness; however, the augmentation caused by the negative stretch was consistently lower than that caused by the positive stretch. Additionally, we proposed a computational model that predicts the perceptual effects based on the preferred directions of the stimulated mechanoreceptors. When examining the grip force, we found that participants applied higher grip forces during the interactions with positive skin-stretch in comparison to the negative skin-stretch, which is consistent with the perceptual results. These results may be useful in tactile technologies for wearable haptic devices, teleoperation, and robot-assisted surgery.

Congruent visual cues speed dynamic motor adaptation.

Motor adaptation to novel dynamics occurs rapidly using sensed errors to update the current motor memory. This adaption is strongly driven by proprioceptive and visual signals that indicate errors in the motor memory. Here, we extend this previous work by investigating whether the presence of additional visual cues could increase the rate of motor adaptation, specifically when the visual motion cue is congruent with the dynamics. Six groups of participants performed reaching movements while grasping the handle of a robotic manipulandum. A visual cue (small red circle) was connected to the cursor (representing the hand position) via a thin red bar. After a baseline, a unidirectional (3 groups) or bidirectional (3 groups) velocity-dependent force field was applied during the reach. For each group, the movement of the red object relative to the cursor was either congruent with the force field dynamics, incongruent with the force field dynamics, or constant (fixed distance from the cursor). Participants adapted more to the unidirectional force fields than to the bidirectional force field groups. However, across both force fields, groups in which the visual cues matched the type of force field (congruent visual cue) exhibited higher final adaptation level at the end of learning than the control or incongruent conditions. In all groups, we observed that an additional congruent cue assisted the formation of the motor memory of the external dynamics. We then demonstrate that a state estimation-based model that integrates proprioceptive and visual information can successfully replicate the experimental data.NEW & NOTEWORTHY We demonstrate that adaptation to novel dynamics is stronger when additional online visual cues that are congruent with the dynamics are presented during adaptation, compared with either a constant or incongruent visual cue. This effect was found regardless of whether a bidirectional or unidirectional velocity-dependent force field was presented to the participants. We propose that this effect might arise through the inclusion of this additional visual cue information within the state estimation process.

Stability of inverted pendulum reveals transition between predictive control and impedance control in grip force modulation.

During object manipulation, our sensorimotor sys-tem needs to represent the objects dynamics in order to better control it. This is especially important in the case of grip force control where small forces can cause the object to slip from our fingers, and excessive forces can cause fatigue or even damage the object. While the tradeoff between these two constraints is clear for stable objects, such as lifting a soda can, it is less clear how the sensorimotor system adjusts the grip force for unstable objects. For this purpose, we measured the change in the grip force of individual human participants while they stabilize five different lengths of an inverted pendulum. These lengths set different dynamics of the pendulum, ranging in their degree of controllability. We observed two main states during such manipulation, a marginally stable state of the pendulum and a stabilization state in which participants acted to stabilize the system. While during the stabilization state participants increased their applied grip force, for the stable state we observed a mixed behaviour. For small and less controllable pendulums, grip force increased while for larger pendulums, participants could modulate the the grip force according to the anticipated load forces. Based on these results, we suggest that the pendulum dynamics change the control strategy between predictive control and impedance control.

Learning of Dexterous Object Manipulation in a Virtual Reality Environment.

Humans have unrivalled abilities to perform dexterous object manipulation. This requires the sensorimotor system to quickly adapt to environmental changes and predictively counter act the external disturbances. Many studies have focused on the anticipatory control of digits with real-world experiments. However, examining manipulation using virtual reality with haptic devices expands the possibilities of investigation. In this work, participants grasped and lifted an inverted T-shaped object in a virtual reality setup. The graspable surface of the object was either constrained to a small area or unconstrained. The position of the object's center of mass changed between blocks, and the participants were asked to minimize the rotation of the object during the lift. Our results show that, consistent with the results of real-world experiments, participants gradually learn to adjust the digit positions and forces to predictively compensate for the torque due to the shifted center of mass prior to liftoff. The only major difference found was that the length of trials needed during the adaptation phase to each condition increased from 3 in real-world to 5 in virtual environment.

Bimanual Manipulation of a Complex Object with Internal Dynamics.

Object manipulation often requires coordination between hands and adaption to the dynamic characteristics of the object. When manipulating the same object, the two hands can have either symmetric or asymmetric impact on the object's trajectory. In this work, we used a bimanual manipulation task of a complex object with internal dynamics to examine how symmetric or scaled-down control of one of the hands affects the coordination between hands. Our result shows that participants are able to quickly adapt to different conditions but the coordination between the two hands changes very little.

Force Control During the Precision Grip Translates to Virtual Reality.

When grasping and manipulating objects we implicitly adapt grip forces according to the physical parameters of the object. We integrate visual, cutaneous, and force feedback to estimate these parameters and adapt our control accordingly. Using virtual reality, both feedback integration and control can be investigated in ways that are not possible using real-life objects. Here, we present our custom-built virtual reality setup and show its validity for use in human studies of fine motor control. Participants grasped and lifted virtual objects with different weights. We show that, consistent with lifting real objects, all participants adapt their grip forces to the object mass, and do so on a trial-by-trial basis. Compared to similar studies with real objects and full feedback, grip forces were increased, and adaptation required more trials. This study successfully demonstrated that grip force control in the precision grip translates to virtual reality. While our setup can be used for similar work in the future, subsequent virtual reality experiments should include a longer adaptation phase compared to classic setups.

Visual Feedback Weakens the Augmentation of Perceived Stiffness by Artificial Skin Stretch.

Tactile stimulation devices are gaining popularity in haptic science and technology-they are lightweight, low-cost, can be wearable, and do not suffer from instability during closed loop interactions with users. Applying tactile stimulation, by means of stretching the fingerpad skin concurrently with kinesthetic force feedback, has been shown to augment the perceived stiffness during interactions with elastic objects. However, to date, the perceptual augmentation due to artificial skin-stretch was studied in the absence of visual feedback. In this article, we tested whether this perceptual augmentation is robust when the stretch is applied in combination with visual displacement feedback. We used a forced-choice stiffness discrimination task with four conditions: force feedback, force feedback with skin-stretch, force and visual feedback, and force and visual feedback with skin-stretch. We found that the visual feedback weakens, but does not eliminate, the skin-stretch induced perceptual effect. Additionally, no effect of visual feedback on the discrimination precision was found.

Nanomesh pressure sensor for monitoring finger manipulation without sensory interference.

Monitoring of finger manipulation without disturbing the inherent functionalities is critical to understand the sense of natural touch. However, worn or attached sensors affect the natural feeling of the skin. We developed nanomesh pressure sensors that can monitor finger pressure without detectable effects on human sensation. The effect of the sensor on human sensation was quantitatively investigated, and the sensor-applied finger exhibits comparable grip forces with those of the bare finger, even though the attachment of a 2-micrometer-thick polymeric film results in a 14% increase in the grip force after adjusting for friction. Simultaneously, the sensor exhibits an extreme mechanical durability against cyclic shearing and friction greater than hundreds of kilopascals.

A bang-bang control model predicts the triphasic muscles activity during hand reaching.

There are numerous ways to reach for an apple hanging from a tree. Yet, our motor system uses a specific muscle activity pattern that features activity bursts and silent periods. We suggest that these bursts are an evidence against the common view that the brain controls the commands to the muscles in a smooth continuous manner. Instead, we propose a model in which a motor plan is transformed into a piecewise-constant control signal that is low-pass filtered and transmitted to the muscles with different muscle-specific delays. We use a Markov chain Monte Carlo (MCMC) method to identify transitions in the state of the muscles following initial activation and show that fitting a bang-bang control model to the kinematics of movement predicts these transitions in the state of the muscles. Such a bang-bang controller suggests that the brain reduces the complexity of the problem of ballistic movements control by sending commands to the muscles at sparse times. Identifying this bang-bang controller can be useful to develop efficient controllers for neuroprostheses and other physical human-robot interaction systems.NEW & NOTEWORTHY While ballistic hand reaching movements are characterized by smooth position and velocity signals, the activity of the muscles exhibits bursts and silent periods. Here, we propose that a model based on bang-bang control provides the link between the abrupt changes in the muscle activity and the smooth reaching trajectory. Using bang-bang control instead of continuous control may simplify the design of prostheses and other physical human-robot interaction systems.

Stretching the skin immediately enhances perceived stiffness and gradually enhances the predictive control of grip force.

When manipulating objects, we use kinesthetic and tactile information to form an internal representation of their mechanical properties for cognitive perception and for preventing their slippage using predictive control of grip force. A major challenge in understanding the dissociable contributions of tactile and kinesthetic information to perception and action is the natural coupling between them. Unlike previous studies that addressed this question either by focusing on impaired sensory processing in patients or using local anesthesia, we used a behavioral study with a programmable mechatronic device that stretches the skin of the fingertips to address this issue in the intact sensorimotor system. We found that artificial skin-stretch increases the predictive grip force modulation in anticipation of the load force. Moreover, the stretch causes an immediate illusion of touching a harder object that does not depend on the gradual development of the predictive modulation of grip force.

A Technique for Measuring Visuomotor Feedback Contributions to the Control of an Inverted Pendulum.

We developed a new technique to measure the contributions of rapid visuomotor feedback responses to the stabilization of a simulated inverted pendulum. Human participants balanced an inverted pendulum simulated on a robotic manipulandum. At a random time during the balancing task, the visual representation of the tip of the pendulum was shifted by a small displacement to the left or right while the motor response was measured. This response was either the exerted force against a fixation position, or the motion to re-stabilize the pendulum in the free condition. Our results demonstrate that rapid involuntary visuomotor feedback responses contribute to the stabilization of the pendulum.

Feedback Delay Changes the Control of an Inverted Pendulum.

We recently developed a simulated inverted pendulum in order to examine human sensorimotor control strategies for stabilization. This simulated system allows us to manipulate the visual and haptic feedback independently from the physical dynamics of the task. Here we examine the effect of sensory delay in a balancing task. Human participants attempted to balance an inverted pendulum (simulated on a robotic manipulandum) with three different added delays (25, 50, and 75 ms), where the same delay was added to both the visual and haptic feedback. Increasing sensory delays decreased the ability of the participants to stabilize the pendulum. Investigation into the online control of the pendulum showed that with longer delays participants reduced their movement frequency but increased the amplitudes of their corrections.

LQG framework explains performance of balancing inverted pendulum with incongruent visual feedback.

Successful manipulation of objects requires forming internal representations of the object dynamics. To do so, the sensorimotor system uses visual feedback of the object movement allowing us to estimate the object state and build the representation. One way to investigate this mechanism is by introducing a discrepancy between the visual feedback about the object's movement and the actual movement. This causes a decline in the ability to accurately control the object, shedding light about possible factors influencing the performance. In this study, we show that an optimal feedback control framework can account for the performance and kinematic characteristics of balancing an inverted pendulum when visual feedback of pendulum tip did not represent the actual pendulum tip. Our model suggests a possible mechanism for the role of visual feedback on forming internal representation of objects' dynamics.

Controller Gains of an Inverted Pendulum are Influenced by the Visual Feedback Position.

In this study we experimentally test and model the control behavior of human participants when controlling inverted pendulums of different dynamic lengths, and with visual feedback of varying congruence to these dynamic lengths. Participants were asked to stabilize the inverted pendulum of L = 1 m and L = 4 m, with visual feedback shown at various distances along the pendulum. We fit a family of linear models to the control input (cart velocity) applied by participants. We further tested the models by predicting this control input for a pendulum with dynamic length L = 2 m and comparing the prediction to the experimental data. We show that the sum of proportional error correction and a term inversely proportional to visual feedback gain can well describe the control in human participants.

Force feedback delay affects perception of stiffness but not action, and the effect depends on the hand used but not on the handedness.

Interaction with an object often requires the estimation of its mechanical properties. We examined whether the hand that is used to interact with the object and their handedness affected people's estimation of these properties using stiffness estimation as a test case. We recorded participants' responses on a stiffness discrimination of a virtual elastic force field and the grip force applied on the robotic device during the interaction. In half of the trials, the robotic device delayed the participants' force feedback. Consistent with previous studies, delayed force feedback biased the perceived stiffness of the force field. Interestingly, in both left-handed and right-handed participants, for the delayed force field, there was even less perceived stiffness when participants used their left hand than their right hand. This result supports the idea that haptic processing is affected by laterality in the brain, not by handedness. Consistent with previous studies, participants adjusted their applied grip force according to the correct size and timing of the load force regardless of the hand that was used, the handedness, or the delay. This suggests that in all of these conditions, participants were able to form an accurate internal representation of the anticipated trajectory of the load force (size and timing) and that this representation was used for accurate control of grip force independently of the perceptual bias. Thus these results provide additional evidence for the dissociation between action and perception in the processing of delayed information. NEW & NOTEWORTHY Introducing delay to force feedback during interaction with an elastic force field biases the perceived stiffness of the force field. We show that this bias depends on the hand that was used for probing but not on handedness. At the same time, both left-handed and right-handed participants adjusted their applied grip force while using either their left or right hands in anticipation of the correct magnitude and timing despite the delay in load force.

The Mechanical Representation of Temporal Delays.

When we knock on a door, we perceive the impact as a collection of simultaneous events, combining sound, sight, and tactile sensation. In reality, information from different modalities but from a single source is flowing inside the brain along different pathways, reaching processing centers at different times. Therefore, interpreting different sensory modalities which seem to occur simultaneously requires information processing that accounts for these different delays. As in a computer-based robotic system, does the brain use some explicit estimation of the time delay, to realign the sensory flows? Or does it compensate for temporal delays by representing them as changes in the body/environment mechanics? Using delayed-state or an approximation for delayed-state manipulations between visual and proprioceptive feedback during a tracking task, we show that tracking errors, grip forces, and learning curves are consistent with predictions of a representation that is based on approximation for delay, refuting an explicit delayed-state representation. Delayed-state representations are based on estimating the time elapsed between the movement commands and their observed consequences. In contrast, an approximation for delay representations result from estimating the instantaneous relation between the expected and observed motion variables, without explicit reference to time.

State-Based Delay Representation and Its Transfer from a Game of Pong to Reaching and Tracking.

To accurately estimate the state of the body, the nervous system needs to account for delays between signals from different sensory modalities. To investigate how such delays may be represented in the sensorimotor system, we asked human participants to play a virtual pong game in which the movement of the virtual paddle was delayed with respect to their hand movement. We tested the representation of this new mapping between the hand and the delayed paddle by examining transfer of adaptation to blind reaching and blind tracking tasks. These blind tasks enabled to capture the representation in feedforward mechanisms of movement control. A Time Representation of the delay is an estimation of the actual time lag between hand and paddle movements. A State Representation is a representation of delay using current state variables: the distance between the paddle and the ball originating from the delay may be considered as a spatial shift; the low sensitivity in the response of the paddle may be interpreted as a minifying gain; and the lag may be attributed to a mechanical resistance that influences paddle's movement. We found that the effects of prolonged exposure to the delayed feedback transferred to blind reaching and tracking tasks and caused participants to exhibit hypermetric movements. These results, together with simulations of our representation models, suggest that delay is not represented based on time, but rather as a spatial gain change in visuomotor mapping.

Stimulation of PPC Affects the Mapping between Motion and Force Signals for Stiffness Perception But Not Motion Control.

How motion and sensory inputs are combined to assess an object's stiffness is still unknown. Here, we provide evidence for the existence of a stiffness estimator in the human posterior parietal cortex (PPC). We showed previously that delaying force feedback with respect to motion when interacting with an object caused participants to underestimate its stiffness. We found that applying theta-burst transcranial magnetic stimulation (TMS) over the PPC, but not the dorsal premotor cortex, enhances this effect without affecting movement control. We explain this enhancement as an additional lag in force signals. This is the first causal evidence that the PPC is not only involved in motion control, but also has an important role in perception that is disassociated from action. We provide a computational model suggesting that the PPC integrates position and force signals for perception of stiffness and that TMS alters the synchronization between the two signals causing lasting consequences on perceptual behavior. SIGNIFICANCE STATEMENT: When selecting an object such as a ripe fruit or sofa, we need to assess the object's stiffness. Because we lack dedicated stiffness sensors, we rely on an as yet unknown mechanism that generates stiffness percepts by combining position and force signals. Here, we found that the posterior parietal cortex (PPC) contributes to combining position and force signals for stiffness estimation. This finding challenges the classical view about the role of the PPC in regulating position signals only for motion control because we highlight a key role of the PPC in perception that is disassociated from action. Altogether this sheds light on brain mechanisms underlying the interaction between action and perception and may help in the development of better teleoperation systems and rehabilitation of patients with sensory impairments.